Proof of Convergence for Evolutionary Policy Iteration under a Sampling Regime
نویسندگان
چکیده
This article extends the evolutionary policy selection algorithm of Chang et al. (2005, 2007), which was designed for use in infinite horizon Markov decision processes (MDPs) with a large action space to a discrete stochastic optimization problem, in an algorithm called Evolutionary Policy Iteration-Monte Carlo (EPI-MC). EPI-MC allows EPI to be used in a setting with a finite decision (action) space and a noisy cost (value) function by introducing a sampling schedule. Convergence of EPI-MC to the optimal decision is proven.
منابع مشابه
Convergence Analysis of Kernel-based On-policy Approximate Policy Iteration Algorithms for Markov Decision Processes with Continuous, Multidimensional States and Actions
Using kernel smoothing techniques, we propose three different online, on-policy approximate policy iteration algorithms which can be applied to infinite horizon problems with continuous and vector-valued states and actions. Using Monte Carlo sampling to estimate the value function around the post-decision state, we reduce the problem to a sequence of deterministic, nonlinear programming problem...
متن کاملA Comparison of Iterated Optimal Stopping and Local Policy Iteration for American Options Under Regime Switching
A theoretical analysis tool, iterated optimal stopping, has been used as the basis of a numerical algorithm for American options under regime switching [19]. Similar methods have also been proposed for American options under jump diffusion [3] and Asian options under jump diffusion [4]. We show that a re-arrangement of the numerical algorithm in the form of local policy iteration [21, 17] has p...
متن کاملConvergence of the multistage variational iteration method for solving a general system of ordinary differential equations
In this paper, the multistage variational iteration method is implemented to solve a general form of the system of first-order differential equations. The convergence of the proposed method is given. To illustrate the proposed method, it is applied to a model for HIV infection of CD4+ T cells and the numerical results are compared with those of a recently proposed method.
متن کاملVariational Iteration Method for Free Vibration Analysis of a Timoshenko Beam under Various Boundary Conditions
In this paper, a relatively new method, namely variational iteration method (VIM), is developed for free vibration analysis of a Timoshenko beam with different boundary conditions. In the VIM, an appropriate Lagrange multiplier is first chosen according to order of the governing differential equation of the boundary value problem, and then an iteration process is used till the desired accuracy ...
متن کاملConvergence theorems of an implicit iteration process for asymptotically pseudocontractive mappings
The purpose of this paper is to study the strong convergence of an implicit iteration process with errors to a common fixed point for a finite family of asymptotically pseudocontractive mappings and nonexpansive mappings in normed linear spaces. The results in this paper improve and extend the corresponding results of Xu and Ori, Zhou and Chang, Sun, Yang and Yu in some aspects.
متن کامل